Docofiiiif iii^iliii 



wn m aoi 

time 



ii 006 116 



Apr 75 

*Jp*i S>t«69nt#d at< th« i^nnual Heating o£ the 

Aierican Educational B#8tateli Association 
(Washington* D.C.y April 1975) 

l!F-$0.76 HC-$1.50 nUS POSTAGE 
EducationiVl Object ivess ^Educational Besaazrch; 
♦Btaluatiooi "^Evaluation Hethods; 4>Hi9h«}: Education; 
♦Hsasuresent Techniques; Questionnaires; Beseacch 
Criteria; Teacher Attitudes 
IDEM71FXSBS ^Delphi Technique 

ABaTBACT 

The Delphi technique is being used by sore 
institutions of higher education as a tool to obtain consensus. A fen 
of the areas in vhich it has been eaployed are identifying 
educational goals and objectives# curricului and caipus planning, 
studies oi cost^ef fectiveness and cost-benefit analysis* evaluation 
procedures, and studies of resource allocation. While sost Delphi 
studies obtain convergence of opinion after tvo or sore rounds* none 
of the studies investigated the peraanency of the opinion 
convergence. To investigate this question* 26 faculty who had. 
participated in an earlier Delphi study of goals* were given the 
identical questionnaire one year later. This questionnaire asked 
their perceptions of the isportance given to different goals by their 
institution as veil as what degree of iaportance they think should be 
attached to each goal. In both analyses it was found that ratings of 
the seccnd questionnaire were siailar to the initial Delphi 
question«iaire. In addition* there was a significant interaction 
between these different scores and the goal areas. Khile this study 
does not provide any definite answer to the original question of the 
peraanency or response changes as a result of the Delphi technique* 
it does indicate that this is likely to be a fertile area for 
additional research. (Author/PG) 
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bow purtnau^nt are rosponae chdnge» a x&mV. qS ompioying the Del- 
phi technique? Thi» queHtlon led to the more specific question of this 
Htudy: It tauuity who had participated in a Delphi study of their insti" 
tutlon's goals responded to the same goals instrument one year later» 
would their responses be closer to the first or last round? Responses 
to the questionnaire given one year later minus responses to the initial 
Delphi questionnaire were compared with responses to the questionnaire 
given one year later minus responses to the final Delphi questionnaire. 
With very few exceptions » it was found that the latest responses were sig- 
nificantly closer (p<*05} to the initial Delphi responses. 

As Indicated In an article by Judd in College and University Business 
(1972), the Delphi technique is being used by more and more institutions 
of higher etiucation as a tool to obtain connensus* A few of the areas in 
which it has been employed to encourage consensus are identifying college* 
universitywide« and statewide educational goals and objectives; curri- 
cvU.m and campus plaimingi studies of 6ost effectiveness and cost-benefit 
analysis; rating scales and other evaluation procedures; educational goals 
and objectives for the future; and studies of resource allocation. 

While most of the Delphi studies obtained convergence of opinion af- 
ter two or more rounds, none of the above studies investigates the per- 
manency of this opinion convergence. When consensus is achieved, does it 
represent a real shift in attitude or perception on the part of the indi- 
viduals who changed their responses or are the changes only of a temporary 
nature? 

Method 

To investigate this question, twenty-six faculty who had participated 
in a Delphi study of goals conducted by the author (Uhl, 1971) were given 
the identical questionnaire one year later. This questionnaire asks for 
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tiietr ptestinfe iJiifacnitions ot the degree importanau given to iiifferieint 
gitiUH by thiiiv ins ci t ut ion as wail a& what dogras of importance tUay think 
Sli^iHM M atto^iUed to aach goal, thuii both percept ions and attitudigid are 
aa»a»Btid. In tlm originai iitudy» participants converged in moat goal areas 
from rather diverse opinions on the first round to general agreofflent on 
the third round. The question of interest here is whether their responses 
to the first or last round of the Delphi will be closest to the responses 
one year later. If the responses are closer to the first round» it would 
provide support for the argument that the Delphi process (as used in this 
goals study) does not have a lasting effect* whereas the opposite results 
would provide support to the hypothesis that a real shift in attitude and/o 

« 

perception does occur. 

Each participant's response to the 110 items in the questionnaire 
given one year later were compared with that individual's responses to the 
110 items On the initial questionnaire and on the final questionnaire. 
Difference scores were calculated and compared (responses to the question- 
naire given one year later minus responses to the initial Delphi question- 
naire were compared with responses to the questionnaire given one year 
later minus responses to the final Delphi questionnaire). Since the items 
of the questionnaire were divided into 18 goal areas, it was possible to 
obtain difference scores per goal area by summing the appropriate item 
difference scores. The statistical analyses compared the two difference 
scores for the 18 goal areas. Tills involved two separate analyses (using 
Linquist's treatment by treatment by subjects design): one used the diff- 
erence scores from ratings of present importanoe while the other used diff- 
erence scores obtained from ratings of preferred importance. 



In bot;h amiXyiia'a it wab iomxd t,Mt tatlugB of Imparlance obtained one 
y«ar aCtt^r the luiliUii, study had baon uotnpiatad wete luore siittllai* to th<i 
rafelttga givtm on tU« Initial rather than the final Delphi questionnaire. 
(Tlw two dlt'fertmea scores were significantly different at the .05 level.) 
tn addition there was a significant interaction between these difference 
scores and the goal areas. Upon examination of the cell means, it was 
touud that this interaction was caused by a few goal areas (three of the 
18 goal areas in the present importance analysis - Personal Development, 
Vocational Preparation, and Research •* and four of the 18 in the preferred 
importance analysis - Personal Development, Research, Social Conscience, 
and Esprit and Quality of Life) for which the responses to the questionnaire 
given one year later were more similar to those responses to the final 
than the initial questionnaire. 

Discussion 

The content of the few goal areas which were exceptions were reviewed 
but did not offer an explanation for their differing results. In general, 
the results provide support to the hypothesis that, in this goals study, 
the changes in opinion as a result of the Delphi technique were only of a 
temporary nature. However, before these findings can be generalized, one 
must consider the limitations of the present study. 

1. The subject of the study were present and preferred importance 
ratings of goals. It is possible that similar results would not be ob~ 
tained with other subject r fitter. 

2. The participants were limited to faculty. 
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4« ^iilG thi't tm^itH of the urlginal Di^lphi godla Btudy were uaed in 
iu^tita^ioaai planning, m^t ot Hia l^culfey iii(4uded in i;his £oliuw-up 
u&uUy did not participate in th^^ process} of planningy md therefor^i have 
not \iom in contact with tU^ results of the Delphi study during the year 
prior to this foilow-up study. Different results miglit have been obtained 
if the faculty were using the results of the Delphi s:udy or if the ad-* 
minis tratiors who were responsible for the planning ht)d been included in 
this study. 

5. If the follow-up study had been performed af^er a shorter time 
interval(e.g.six months), different results may have been obtained. 

6. The study was conducted at one institution. Institutions with 
different characteristics might obtain different results. 

While this study does not provide any definitive answer to the ori- 
ginal question of the permanency of response changes as a result of the 
Delphi technique, it does indicate that this is likely to be a fertile 
area for additional research. Certain variables have been identified 
which should be considered in such studies. 



Juda, R.C. "For^casfeljjg m Gomiinmu Gathering, Delphi (irowtj Uis to Gol- 
l«g© Keeds." calleg^ and Univegaity ftuaifte^a . My, 1972 » pi>, 35^*37. 

Uhl, N4^ TtlgntU'yin^ tnatitutional (ioala ? E ncouraging Conyaifk€*nce of 

"ghise Eduuation, 1971. 



7 



